Minimax redundancy for the class of memoryless sources

Let Xn = (X1; ; Xn) be a memoryless source with unknown distribution on a finite alphabet of size k. We identify the asymptotic minimax coding redundancy for this class of sources, and provide a sequence of asymptotically minimax codes. Equivalently, we determine the limiting behavior of the minimax relative entropy minQ maxP D(PX kQX ), where the maximum is over all independent and identically distributed (i.i.d.) source distributions and the minimum is over all joint distributions. We show in this paper that the minimax redundancy minus ((k 1)=2) log (n=(2 e)) converges to log det I( ) d = log ( (1=2)= (k=2)), where I( ) is the Fisher information and the integral is over the whole probability simplex. The Bayes strategy using Jeffreys’ prior is shown to be asymptotically maximin but not asymptotically minimax in our setting. The boundary risk using Jeffreys’ prior is higher than that of interior points. We provide a sequence of modifications of Jeffreys’ prior that put some prior mass near the boundaries of the probability simplex to pull down that risk to the asymptotic minimax level in the limit.